Mobile device edge view display insert

ABSTRACT

A mobile communication device may include one or more cameras located on edges of the mobile communication device. The mobile communication device may further include logic configured to obtain image data from at least one of the one or more cameras; detect a change in an environment of the mobile communication device based on the obtained image data; and provide image data from the at least one of the one or more cameras in a display insert window on a display of the mobile communication device.

BACKGROUND INFORMATION

Mobile communication devices are used in many situations. For example, auser may use a mobile communication device for sending text messages,browsing the internet, or watching a video. Such activities may requirea user to look down on the screen of the mobile communication device.Often, users may have their attention focused on the mobilecommunication device without being aware of their environment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary environment according toan implementation described herein;

FIGS. 2A-2D are diagrams illustrating exemplary components of the mobilecommunication device of FIG. 1;

FIG. 3 is a diagram illustrating the field of view of a user and a fieldof view that may be available to a mobile communication device of theuser;

FIG. 4 is a diagram of exemplary components of the mobile communicationdevice of FIG. 1;

FIG. 5 is a diagram of exemplary functional components of the mobilecommunication device of FIG. 1;

FIG. 6 is a diagram of exemplary components of the environment analysissystem of FIG. 1;

FIG. 7 is a diagram of exemplary functional components of theenvironment analysis system of FIG. 1;

FIG. 8 is a flowchart for providing a display insert based on one ormore edge view cameras according to an implementation described herein;

FIG. 9 is a flowchart for relating images from one or more edge viewcameras to user contacts according to an implementation describedherein;

FIG. 10 is a diagram of a first exemplary scenario according to animplementation described herein;

FIG. 11 is a diagram of a second exemplary scenario according to animplementation described herein;

FIG. 12 is a diagram of a third exemplary scenario according to animplementation described herein; and

FIG. 13 is a diagram of a fourth exemplary scenario according to animplementation described herein.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings identify the same orsimilar elements.

When a user is looking down on the display of a mobile communicationdevice, the user may not be aware of the environment around the user,and may not notice a change in the environment, such as an approachingvehicle or person. Implementations described herein relate to a mobilecommunication device edge view display insert. The mobile communicationdevice may include one or more edge cameras located on edges of themobile communication device. An edge may correspond to a plane that isperpendicular in relation to a plane of a display of the mobilecommunication device. For example, the mobile communication device mayinclude a top edge camera located on a top edge of the mobilecommunication device. Thus, when the mobile communication device is in ahorizontal, or near horizontal, position as the user is looking down onthe display, the top edge camera may point forward and may capture afield of view in front of the user. Additionally or alternatively, themobile communication device may include one or more side edge cameraslocated on the side edges of the mobile communication device, such as aleft edge camera and/or a right edge camera, which may capture a fieldof view to the user's side.

The mobile communication device may include logic configured to obtainimage data from one or more cameras, detect a change in an environmentof the mobile communication device based on the obtained image data,generate an alert informing a user of the mobile communication device ofthe detected change in the environment, and provide image data from atleast one of the cameras in a display insert window (e.g., a smallpop-up display window) on a display of the mobile communication device.The one or more cameras may include edge cameras located on edges of themobile communication device and/or a back camera located on the back ofthe mobile communication device. Particular camera to monitor theenvironment may be selected based on the orientation of the mobilecommunication device. Furthermore, in some implementations, the one ormore cameras may include a remote camera that communicates with themobile communication device using a wireless connection, such as a dashmounted camera in a vehicle. Moreover, image data obtained from multiplecameras may be combined to produce a single image or video stream, suchas a panoramic view image or video stream.

Image data from the selected one or more cameras may be displayed in adisplay insert window on a display of the mobile communication device.Thus, the display insert window may provide images and/or a video feedof the environment. The user may select to continuously display thedisplay insert window. Alternatively, the user may select to have thedisplay insert window appear in response to a detected change in theenvironment.

A change in the environment may be detected by detecting motion, achange in light intensity, and/or a change in a color profile of theenvironment based on the image data. A change may also be detected fromanother type of change in the image data or other collected data, suchas audio data captured by a microphone. In some implementations, one ormore of the cameras may include a zoom lens assembly and the zoom lensassembly may be focused on an area of the environment in which thechange is detected.

In some implementations, the mobile communication device may beconfigured to perform pattern recognition on captured images, to detecta change in the environment. The mobile communication device mayidentify an object, entity, or event in the environment based on thepattern recognition. For example, the mobile communication device maydetect a moving vehicle and may inform the user that the moving vehicleis approaching the user.

As another example, the mobile communication device may detect a person,may perform facial recognition to identify the person, and may retrievecontact information associated with the user based on theidentification. The contact information may be obtained from the user'saddress book, a social media account of the user, and/or anotherdatabase of contacts. The user may be informed that the person has beenidentified and/or that the contact information has been obtained.Furthermore, in some implementations a calendar event associated withthe contact may be identified and information relating to the calendarevent may be provided to the user.

FIG. 1 is a diagram of an exemplary environment 100 in which the systemsand/or methods described herein may be implemented. As shown in FIG. 1,environment 100 may include a mobile communication device 110, a network120, and an environment analysis system 130.

Mobile communication device 110 may include any mobile communicationdevice that includes a display and a camera. For example, mobilecommunication device 110 may include a portable communication device(e.g., a mobile phone, a smart phone, a phablet device, a globalpositioning system (GPS) device, and/or another type of wirelessdevice); a personal computer or workstation; a server device; a laptop,tablet, or another type of portable computer; a media playing device; aportable gaming system; and/or any type of mobile device withcommunication capability. Mobile communication device 110 may includeone or more edge cameras located on an edge of the mobile communicationdevice 110 and may be configured to provide a display insert of a scenefrom the environment based on a detected change in the environment.

Network 120 may enable mobile communication device 110 and environmentanalysis system 130 to communicate with each other. Network 120 mayinclude one or more circuit-switched networks and/or packet-switchednetworks. For example, network 120 may include a local area network(LAN), a wide area network (WAN), a metropolitan area network (MAN), aPublic Switched Telephone Network (PSTN), an ad hoc network, anintranet, the Internet, a fiber optic-based network, a wireless network,and/or a combination of these or other types of networks.

Environment analysis system 130 may include one or more devices, such asserver devices, that analyze the environment of mobile communicationdevice 110 based on images, audio signals, and/or videos obtained bymobile communication device 110. For example, environment analysissystem 130 may perform image processing, image/pattern recognition(e.g., facial recognition), contact identification (e.g., identifying aperson that is in a contact list of the user), and/or othercomputationally demanding processes.

Although FIG. 1 shows exemplary components of environment 100, in otherimplementations, environment 100 may include fewer components, differentcomponents, differently arranged components, or additional componentsthan the ones depicted in FIG. 1. Additionally or alternatively, one ormore components of environment 100 may perform functions described asbeing performed by one or more other components of environment 100.

FIGS. 2A-2D are diagrams of an exemplary mobile communication device 110according to an implementation. FIG. 2A illustrates a top view of mobilecommunication device 110; FIG. 2B illustrates a front view of mobilecommunication device 110; FIG. 2C illustrates a side view of mobilecommunication device 110; and FIG. 2D illustrates a back view of mobilecommunication device 110. As shown in FIGS. 2A-2D, mobile communicationdevice 110 may include a housing 210, a touchscreen 220, a microphone230, a speaker 240, a back camera 250, a top edge camera 260, a leftedge camera 270, and a right edge camera 280.

Housing 210 may enclose mobile communication device 110 and may protectthe components of mobile communication device 110 from the outsideelements (e.g., moisture, dirt, etc.). Touchscreen 220 may include adisplay device and an input device configured to detect a user's touch.While FIGS. 2A-2D illustrate mobile communication device 110 with atouchscreen, in other implementations, mobile communication device 110may not necessarily include a touchscreen. For example, mobilecommunication device 110 may include a display and a keyboard and/orkeypad. Touchscreen 220 may include a liquid crystal display (LCD), anelectronic ink display (e.g., an electrophoretic display), anelectroluminescent display, and/or another type of display. Furthermore,touchscreen 220 may include touch sensors, such as capacitive sensors(e.g., surface capacitive sensors, projected capacitive touch sensors,etc.), resistive sensors (e.g., analog resistive sensors, digitalresistive sensors, etc.), optical sensors (e.g., optical imagingsensors, rear diffused illumination sensors, infrared grid sensors,diffused surface illumination sensors, etc.), acoustic wave sensors(e.g., surface acoustic wave sensors, bending wave sensors, etc.),and/or another type of touch sensors. Furthermore, touchscreen 220 mayinclude sensors to sense an amount of force being applied to touchscreen220, such as piezoresistive sensors.

Microphone 230 may receive audio signals and convert the received audiosignals to electrical signals for mobile communication device 110.Speaker 240 may receive electrical signals from within mobilecommunication device 110 and generate audio signals based on thereceived electrical signals.

Mobile communication device 110 may include additional sensors (notshown in FIGS. 2A-2D). For example, mobile communication device 110 mayinclude one or more tilt sensors, such as accelerometers and/orgyroscopes, configured to sense a tilt, position, and/or orientation ofmobile communication device 110 in space; one or more Global PositioningSystem (GPS) receivers; one or more magnetic field sensors (e.g., tosense the magnetic North); motion detection sensors to sense motion inthe vicinity of mobile communication device 110; and/or other types ofsensors.

Back camera 250, top edge camera 260, left edge camera 270, and rightedge camera 280 (referred to herein collectively as “the cameras”) mayeach include a visible light camera, an infrared light camera, anultraviolet light camera, and/or another type of camera. Back camera 250may be located on the back of mobile communication device 110 and may beconfigured to capture a field of view in front of the user when mobilecommunication device 110 is held in a vertical position. Top edge camera260 may be located on the top edge of mobile communication device 110and may be configured to capture a field of view of in front of the userwhen mobile communication device 110 is in a horizontal position. Leftedge camera 270 may be located on the left edge of mobile communicationdevice 110 and may be configured to capture a field of view to the leftof the user. Right edge camera 280 may be located on the right edge ofmobile communication device 110 and may be configured to capture a fieldof view to the right of the user.

In some implementations, at least some of the cameras may include afisheye camera that includes a wide angle lens, enabling a camera tocapture a wide angle image (e.g., a 180 degree view image) of theenvironment. In some implementations, at least some of the cameras mayinclude a mechanical zoom assembly that enables a particular camera tozoom in on an area in the field of view of the camera.

Although FIGS. 2A-2D show exemplary components of mobile communicationdevice 110, in other implementations, mobile communication device 110may include fewer components, different components, differently arrangedcomponents, or additional components than those depicted in FIGS. 2A-2D.Additionally or alternatively, one or more components of mobilecommunication device 110 may perform functions described as beingperformed by one or more other components of mobile communication device110. As an example, in some implementations, mobile communication device110 may include top edge camera 260 and may not include left edge camera270 and right edge camera 280. As another example, in someimplementations, mobile communication device 110 may include a camera onthe bottom edge of mobile communication device 110. As yet anotherexample, in some implementations, mobile communication device 110 mayinclude multiple cameras along a particular edge.

FIG. 3 is a diagram of a user with a mobile communication device 110.FIG. 3 illustrates the field of view of the user and a field of viewthat may be available to mobile communication device 110. As shown inFIG. 3, when user 310 is holding mobile communication device 110 in ahorizontal position, the top edge of mobile communication device 110 ispointing to the environment in front of user 310. Thus, while the user'sfield of view 320 is directed to the screen of mobile communicationdevice 110, top edge camera 260 may be pointed forward. Thus, an imageof the environment in front of the user is within field of view 330 oftop edge camera 260, allowing top edge camera 260 to capture the image.

FIG. 4 is a diagram illustrating example components of a mobilecommunication device 110 according to an implementation describedherein. As shown in FIG. 4, mobile communication device 110 may includea processing unit 410, a memory 420, a user interface 430, acommunication interface 440, an antenna assembly 450, and camera logic460.

Processing unit 410 may include one or more processors, microprocessors,application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), and/or other processing logic. Processing unit 410may control operation of mobile communication device 110 and itscomponents.

Memory 420 may include a random access memory (RAM) or another type ofdynamic storage device, a read only memory (ROM) or another type ofstatic storage device, a removable memory card, and/or another type ofmemory to store data and instructions that may be used by processingunit 410.

User interface 430 may allow a user to input information to mobilecommunication device 110 and/or to output information from mobilecommunication device 110. Examples of user interface 430 may include aspeaker to receive electrical signals and output audio signals (e.g.,speaker 240); a camera to receive image and/or video signals and outputelectrical signals; a microphone to receive sounds and output electricalsignals (e.g., microphone 230); buttons (e.g., a joystick, controlbuttons, a keyboard, or keys of a keypad) and/or a touchscreen toreceive control commands (e.g., touchscreen 220); a display, such as anLCD, to output visual information (e.g., touchscreen 220); an actuatorto cause mobile communication device 110 to vibrate; and/or any othertype of input or output device.

Communication interface 440 may include a transceiver that enablesmobile communication device 110 to communicate with other devices and/orsystems via wireless communications (e.g., radio frequency, infrared,and/or visual optics, etc.), wired communications (e.g., conductivewire, twisted pair cable, coaxial cable, transmission line, fiber opticcable, and/or waveguide, etc.), or a combination of wireless and wiredcommunications. Communication interface 440 may include a transmitterthat converts baseband signals to radio frequency (RF) signals and/or areceiver that converts RF signals to baseband signals. Communicationinterface 440 may be coupled to antenna assembly 450 for transmittingand receiving RF signals.

Communication interface 440 may include a logical component thatincludes input and/or output ports, input and/or output systems, and/orother input and output components that facilitate the transmission ofdata to other devices. For example, communication interface 440 mayinclude a network interface card (e.g., Ethernet card) for wiredcommunications and/or a wireless network interface (e.g., a WiFi) cardfor wireless communications. Communication interface 440 may alsoinclude a universal serial bus (USB) port for communications over acable, a Bluetooth™ wireless interface, a radio-frequency identification(RFID) interface, a near-field communications (NFC) wireless interface,and/or any other type of interface that converts data from one form toanother form.

Antenna assembly 450 may include one or more antennas to transmit and/orreceive RF signals. Antenna assembly 450 may, for example, receive RFsignals from communication interface 440 and transmit the signals andreceive RF signals and provide them to communication interface 440.Camera logic 460 may control the cameras of mobile communication device110. For example, camera logic 460 may control the operation of camerasto capture images of the environment. For example, camera logic 460 maycontrol a mechanical zoom assembly of a particular camera. In someimplementations, one or more of the edge cameras may include a tiltmotor for moving the focus of an edge camera in relation to the plane ofmobile communication device 110 and camera logic 460 may be configuredto control the tilt motor. For example, if mobile communication device110 is held by the user at a 45 degree angle, camera logic may controlthe tilt motor to adjust top edge camera 260 to point straight ahead ina horizontal direction.

As described herein, mobile communication device 110 may perform certainoperations in response to processing unit 410 executing softwareinstructions contained in a computer-readable medium, such as memory420. A computer-readable medium may be defined as a non-transitorymemory device. A non-transitory memory device may include memory spacewithin a single physical memory device or spread across multiplephysical memory devices. The software instructions may be read intomemory 420 from another computer-readable medium or from another devicevia communication interface 440. The software instructions contained inmemory 420 may cause processing unit 410 to perform processes that willbe described later. Alternatively, hardwired circuitry may be used inplace of, or in combination with, software instructions to implementprocesses described herein. Thus, implementations described herein arenot limited to any specific combination of hardware circuitry andsoftware.

Although FIG. 4 shows example components of mobile communication device110, in other implementations, mobile communication device 110 mayinclude fewer components, different components, differently arrangedcomponents, or additional components than depicted in FIG. 4.Additionally or alternatively, one or more components of mobilecommunication device 110 may perform the tasks described as beingperformed by one or more other components of mobile communication device110.

FIG. 5 is a diagram illustrating exemplary functional components ofmobile communication device 110 according to an implementation describedherein. The functional components of mobile communication device 110 maybe implemented, for example, via processing unit 410 executinginstructions from memory 420. For example, the functional components ofFIG. 5 may be implemented in an application installed on mobilecommunication device 110. Alternatively, some or all of the functionalcomponents of mobile communication device 110 may be implemented viahard-wired circuitry. As shown in FIG. 5, mobile communication device110 may include a camera interface 510, a pattern recognition module520, a pattern database (DB) 525, an environment analysis systeminterface 530, a contacts interface 540, a social media interface 550, acalendar interface 560, a user interface generator 570, and a usersettings DB 575.

Camera interface 510 may communicate with, and control, camera logic460. Camera interface 510 may determine an orientation of mobilecommunication device 110 in space and may select one or more of thecameras to monitor the environment, based on the determined orientation.In some implementations, camera interface 510 may combine images frommultiple cameras into a single image or video feed, such as a panoramicimage or video feed. Furthermore, camera interface 510 may establish awireless connection with a remote camera, such as a nearby cameradetected with a Bluetooth or NFC connection, or a remote camera whichthe user has configured to provide images and/or video feeds to mobilecommunication device 110 (e.g., a home surveillance camera).

Pattern recognition module 520 may analyze images and/or video feedsfrom the cameras to determine a change in the environment. For example,pattern recognition module 520 may detect movement in the environmentbased on the images, may detect a change in light intensity in theenvironment based on the images, may detecting a change in a colorprofile of the environment based on the images, may detect a change inthe environment based on audio signals obtained via microphone 230,and/or may detect a change in the environment based on another type ofsensor data.

Furthermore, pattern recognition module 520 may analyze the images toidentify a particular pattern. Pattern recognition module 520 may applyimage processing techniques, such as edge detection, object detection,color histogram analysis, light intensity analysis, and/or othertechniques and may generate one or more feature vectors for a particularimage. The feature vectors may be used to identify a particular patternin the image based on information stored in patterns DB 525.

Pattern DB 525 may store particular patterns and correspondinginformation that may be provided to the user. As an example, pattern DB525 may store a pattern associated with a particular type of vehicle inconnection with information relating to the type of vehicle. As anotherexample, pattern DB 525 may store a pattern associated with a personperforming a particular activity along with information relating to theparticular activity. As yet other examples, pattern DB 525 may store apattern associated with a particular level of traffic congestion orspeed of movement, a pattern associated with a particular street sign(e.g., traffic light, walk sign, etc.), a pattern associated with aparticular number of people, and/or another type of pattern.

Environment analysis system interface 530 may communicate withenvironment analysis system 130. For example, in some implementations,environment analysis system interface 530 may provide image and/or videofeed data, and/or information extracted from the image and/or video feeddata, to environment analysis system 130 for analysis. Environmentanalysis system 130 may perform computationally intensive operations,such as image processing, facial recognition etc. Environment analysissystem interface 530 may receive information from environment analysissystem 130, such as results of the analysis.

Contacts interface 540 may interface with a contacts managementapplication and/or contacts database on mobile communication device 540and may identify contact information (herein referred to as “a contact”)associated with a person that may be identified in images and/or videofeeds obtained by the cameras. Social media interface 550 may interfacewith a social media application account associated with the user, mayidentify a contact associated with the user's social media account, andmay obtain information relating to the identified contact from a socialmedia application and/or social media system. Calendar interface 560 mayinterface with a calendar application on mobile communication device 110and may determine whether a calendar event is associated with aparticular contact that has been identified via an image or video feedobtained from the cameras.

User interface generator 570 may generate a user interface that includesa display insert window in response to a detected change in theenvironment and/or in response to the user selecting to open a displayinsert window that includes an image or video feed from a particularcamera of mobile communication device 110 or from a remote camera.Furthermore, user interface generator 570 may generate an audible alert,a vibration alert, and/or another type of alert in response to thedetected change in the environment. Moreover, user interface generator570 may provide information relating to the detected change. Theinformation may have been obtained based on patter analysis, such asinformation identifying a particular entity, object, or event in theenvironment.

User settings DB 575 may store settings selected by the user. Thesettings may indicate, for example, what type of detected changes in theenvironment should trigger an alert, what kind of alert to provide tothe user, whether the user has selected to check whether a particularcontact has been detected, etc.

Although FIG. 5 shows exemplary functional components of mobilecommunication device 110, in other implementations, mobile communicationdevice 110 may include fewer functional components, different functionalcomponents, differently arranged functional components, or additionalfunctional components than depicted in FIG. 5. Additionally oralternatively, one or more functional components of mobile communicationdevice 110 may perform functions described as being performed by one ormore other functional components of mobile communication device 110.

FIG. 6 is a diagram illustrating exemplary components of environmentanalysis system 130 according to an implementation described herein. Asshown in FIG. 6, environment analysis system 130 may include a bus 610,a processor 620, a memory 630, an input device 640, an output device650, and a communication interface 660.

Bus 610 may include a path that permits communication among thecomponents of environment analysis system 130. Processor 620 may includeany type of single-core processor, multi-core processor, microprocessor,latch-based processor, and/or processing logic (or families ofprocessors, microprocessors, and/or processing logics) that interpretsand executes instructions. In other embodiments, processor 620 mayinclude an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), and/or another type of integratedcircuit or processing logic.

Memory 630 may include any type of dynamic storage device that may storeinformation and/or instructions, for execution by processor 620, and/orany type of non-volatile storage device that may store information foruse by processor 620. For example, memory 630 may include a randomaccess memory (RAM) or another type of dynamic storage device, aread-only memory (ROM) device or another type of static storage device,a content addressable memory (CAM), a magnetic and/or optical recordingmemory device and its corresponding drive (e.g., a hard disk drive,optical drive, etc.), and/or a removable form of memory, such as a flashmemory.

Input device 640 may allow an operator to input information intoenvironment analysis system 130. Input device 640 may include, forexample, a keyboard, a mouse, a pen, a microphone, a remote control, anaudio capture device, an image and/or video capture device, atouch-screen display, and/or another type of input device. In someembodiments, environment analysis system 130 may be managed remotely andmay not include input device 640. In other words, environment analysissystem 130 may be “headless” and may not include a keyboard, forexample.

Output device 650 may output information to an operator of environmentanalysis system 130. Output device 650 may include a display, a printer,a speaker, and/or another type of output device. For example,environment analysis system 130 may include a display, which may includea liquid-crystal display (LCD) for displaying content to the customer.In some embodiments, environment analysis system 130 may be managedremotely and may not include output device 650. In other words,environment analysis system 130 may be “headless” and may not include adisplay, for example.

Communication interface 660 may include a transceiver that enablesenvironment analysis system 130 to communicate with other devices and/orsystems via wireless communications (e.g., radio frequency, infrared,and/or visual optics, etc.), wired communications (e.g., conductivewire, twisted pair cable, coaxial cable, transmission line, fiber opticcable, and/or waveguide, etc.), or a combination of wireless and wiredcommunications. Communication interface 660 may include a transmitterthat converts baseband signals to radio frequency (RF) signals and/or areceiver that converts RF signals to baseband signals. Communicationinterface 660 may be coupled to an antenna for transmitting andreceiving RF signals.

Communication interface 660 may include a logical component thatincludes input and/or output ports, input and/or output systems, and/orother input and output components that facilitate the transmission ofdata to other devices. For example, communication interface 660 mayinclude a network interface card (e.g., Ethernet card) for wiredcommunications and/or a wireless network interface (e.g., a WiFi) cardfor wireless communications. Communication interface 660 may alsoinclude a universal serial bus (USB) port for communications over acable, a Bluetooth™ wireless interface, a radio-frequency identification(RFID) interface, a near-field communications (NFC) wireless interface,and/or any other type of interface that converts data from one form toanother form.

As will be described in detail below, environment analysis system 130may perform certain operations relating to analysis of the environmentof mobile communication device 110 based on one or more images, audiosignals, and/or videos obtained by mobile communication device 110.Environment analysis system 130 may perform these operations in responseto processor 620 executing software instructions contained in acomputer-readable medium, such as memory 630. A computer-readable mediummay be defined as a non-transitory memory device. A memory device may beimplemented within a single physical memory device or spread acrossmultiple physical memory devices. The software instructions may be readinto memory 630 from another computer-readable medium or from anotherdevice. The software instructions contained in memory 630 may causeprocessor 620 to perform processes described herein. Alternatively,hardwired circuitry may be used in place of, or in combination with,software instructions to implement processes described herein. Thus,implementations described herein are not limited to any specificcombination of hardware circuitry and software.

Although FIG. 6 shows exemplary components of environment analysissystem 130, in other implementations, environment analysis system 130may include fewer components, different components, additionalcomponents, or differently arranged components than depicted in FIG. 6.Additionally or alternatively, one or more components of environmentanalysis system 130 may perform one or more tasks described as beingperformed by one or more other components of environment analysis system130.

FIG. 7 is a diagram illustrating exemplary functional components ofenvironment analysis system 130. The functional components ofenvironment analysis system 130 may be implemented, for example, viaprocessor 620 executing instructions from memory 630. Additionally oralternatively, some or all of the functional components of environmentanalysis system 130 may be hard-wired. As shown in FIG. 7, environmentanalysis system 130 may include a mobile communication interface 710, apattern recognition module 720, a pattern DB 725, a user identificationmodule 730, a user DB 735, and a social media interface 740.

Mobile communication interface 710 may communicate with mobilecommunication device 110. For example, mobile communication interface710 may obtain image and/or video feed data, and/or informationextracted from image and/or video feed data, from mobile communicationdevice 110 and may return information relating to the obtained dataand/or generated as a result of analysis performed by environmentanalysis system 130, to mobile communication device 110.

Pattern recognition module 720 may analyze data obtained from mobilecommunication device 110 to identify one or more patterns, similar tothe functions described above with respect to pattern recognition module520, and may obtain information relating to the identified patterns frompatterns DB 725. Pattern DB 725 may store particular patterns andcorresponding information that may be provided to a user of mobilecommunication device 110. For example, pattern DB 725 may storeinformation similar to the information described above with respect topattern DB 525.

Person identification module 730 may identify a particular person in animage and/or video feed obtained from mobile communication device 110.For example, person identification module 730 may perform facialrecognition analysis on an obtained image, may generate a facialrecognition feature vector, may identify a particular person in theimage based on the facial recognition feature vector and based oninformation stored in person DB 735, and may provide information fromperson DB 735 to mobile communication device 110.

Person DB 735 may store information identifying particular persons. Forexample, person DB 735 may store a mobile device identifier associatedwith a person, information identifying the particular person, a socialmedia account associated with the person, contacts associated with theperson, a picture of the person, a facial recognition feature vector,privacy settings selected by the person, and/or other informationassociated with the person. Information stored in person DB 735 about aparticular person may be obtained with the particular person'spermission and in accordance with privacy laws and regulations. Socialmedia interface 740 may interface with one or more social media systemsand may obtain information relating to a particular social media accountassociated with a user and/or with a social media account associatedwith a user's contact.

Although FIG. 7 shows exemplary functional components of environmentanalysis system 130, in other implementations, environment analysissystem 130 may include fewer functional components, different functionalcomponents, differently arranged functional components, or additionalfunctional components than depicted in FIG. 7. Additionally oralternatively, one or more functional components of environment analysissystem 130 may perform functions described as being performed by one ormore other functional components of environment analysis system 130.

FIG. 8 is a flowchart for providing a display insert based on one ormore edge view cameras according to an implementation described herein.In one implementation, the process of FIG. 8 may be performed by mobilecommunication device 110. In other implementations, some or all of theprocess of FIG. 8 may be performed by another device or a group ofdevices separate from and/or including mobile communication device 110,such as environment analysis system 130.

The process of FIG. 8 may include determining a device orientation(block 810). For example, camera interface 510 may determine theorientation of mobile communication device using one or more tiltsensors, such as gyroscope sensors, accelerometers, and/or other typesof sensors. Monitoring cameras may be selected based on the deviceorientation (block 820). For example, if mobile communication device 110is in a horizontal position (or within a particular number of degrees ofa horizontal position) and held in portrait mode, top edge camera 260may be used to monitor the environment in front of the user and leftedge camera 270 and right edge camera 280 may be used to monitor theenvironment to the left and right of the user. If mobile communicationdevice 110 is in a horizontal position (or within a particular number ofdegrees of a horizontal position) and held in a landscape mode, eitherleft edge camera 270 or right edge camera 280 may be used to monitor theenvironment in from of the user. If mobile communication device 110 isin a vertical position (or within a particular number of degrees of ahorizontal position), back camera 250 may be used to monitor theenvironment in from of the user. Furthermore, if one or more of the edgecameras include a tilt motor, camera logic 460 may adjust the tilt of anedge camera based on a determined tilt of mobile communication device110.

Data from the selected cameras may be obtained (block 830) and a changein the environment may be detected (block 840). For example, camerainterface 510 and/or pattern recognition module 520 may monitor imagesand/or video feeds from one or more of the selected camera to detect forchanges in the environment. A change in the environment may be detectedif an amount of motion, greater than a motion threshold, is detected.Motion may be detected using a motion detection sensor and/or based onanalyzing a series of sequential captured imaged. A change in theenvironment may be detected if a change in light intensity is detected.For example, pattern recognition module 520 may calculate one or morelight intensity values for captured images or for particular areas ofcaptured images and may determine whether a light intensity value haschanged over multiple captured images by an amount greater than a lightintensity change threshold. A change in the environment may be detectedif a change in a color profile is detected. For example, patternrecognition module 520 may calculate one or more color profile valuesfor captured images or for particular areas of captured images and maydetermine whether a color profile value has changed over multiplecaptured images by an amount greater than a color profile changethreshold. As an example, a street crossing sign may light up and changecolor and an alert may be generated for the user to inform the user thatthe sign has changed and that the user can now cross the street. Achange in the environment may also be detected based on a detected audiosignal captured by microphone 230. In some implementations, camera logic460 may control a mechanical zoom assembly to zoom in on a part of theenvironment associated with the detected change.

A user may be alerted to the detected change (block 850) and image datamay be provided in a display insert on a device display (block 860). Forexample, user interface generator 570 may provide a visual, auditory,and/or vibration alert to the user in response to the detected change.Whether or not an alert is generated may depend on the user settingstored in user settings DB 575. User interface generator 570 maygenerate a display insert window that may appear on touchscreen 220 andcaptured images and/or a video feed from a particular camera may bedisplayed in the generated display insert window. Alternatively, a usermay select that a display insert window be continuously displayed toprovide image data from the one or more selected monitoring cameraswithout requiring any detected changes in the environment.

Information relating to the detected change may be obtained (block 870)and the obtained information may be provided to the user (block 880).For example, pattern analysis may be performed on one or more capturedimages to identify an entity, object or event in the environment. Insome implementations, the pattern analysis may be performed by mobilecommunication device 110. In other implementations, the pattern analysismay be performed by environment analysis system 130. In yet otherimplementations, some of the pattern analysis may be performed by mobilecommunication system 110 and some of the pattern analysis may beperformed by environment analysis system 130. For example, mobilecommunication device 110 may perform feature extraction on capturedimages and may provide feature vectors obtained from the captured imagesto environment analysis system 130. Environment analysis system 110 mayclassify a feature vector as belonging to a particular class associatedwith a particular pattern. As another example, mobile communicationdevice 110 may perform pattern analysis to detect a person in a capturedimage and may send the captured image to environment analysis system 130to perform facial recognition analysis on the detected person.

Whether particular aspects of the pattern analysis are performed bymobile communication device 110 or environment analysis system 130 maydepend upon one or more of an availability of a network connection, abandwidth capacity of a network connection, a type of subscriptionassociated with mobile communication device 110, a processing loadassociated with processing unit 410, and/or other factors.

The performed pattern recognition may be used retrieve informationrelating to an identified object, entity, or event and informationrelating to the identified object, entity, or event may be provided tothe user in connection with the generated display insert window. As anexample, the information may be provided in a caption displayed above orbelow the display insert window. As another example, the information maybe provided as an audible message via microphone 230. As yet anotherexample, the information may be provided as a hyperlink.

FIG. 9 is a flowchart for relating images from one or more edge viewcameras to user contacts according to an implementation describedherein. In one implementation, the process of FIG. 9 may be performed bymobile communication device 110. In other implementations, some or allof the process of FIG. 9 may be performed by another device or a groupof devices separate from and/or including mobile communication device110, such as environment analysis system 130.

The process of FIG. 9 may include determining/detecting whether a personis present in the field of view of the device cameras (block 910). Forexample, pattern recognition module 520 may detect a person present in acaptured image and may provide the captured image to environmentanalysis system 130 for facial recognition analysis. In otherimplementations, the facial recognition analysis may be performed bymobile communication device 110.

Facial recognition may be performed to identify the determined/detectedperson (block 920) and a contact associated with the identified personmay be identified (block 930). As an example, mobile communicationdevice 110 may determine whether the detected person corresponds to acontact associated with the user, based on pictures of contacts storedin the user's contacts database. As yet another example, mobilecommunication device 110 may determine whether the detected personmatches a contact based on pictures of contacts associated with a user'ssocial media account. As yet another example, environment analysissystem 130 may identify the contact based on pictures and/or facialrecognition feature vectors stored in user DB 735.

An event associated with the identified contact may be identified (block940). For example, in some implementations, pattern recognition module520 may interface with a calendar application to identify a calendarevent associated with the contact and may generate a reminder relatingto the calendar event in connection with the identified contact.

A display insert may be generated based on the identified information(block 950). For example, user interface generator 570 may generate adisplay insert window that provides a video feed of the approachingperson, along with a message, informing the user that the identifiedcontact is approaching.

FIG. 10 is a diagram of a first exemplary scenario 1000 according to animplementation described herein. In scenario 1000, user 1010 is lookingdown at mobile communication device 110 while waiting for a bus. Sincemobile communication device 110 is in a horizontal position, top edgecamera 260 may be selected for monitoring the environment and imagescaptures by top edge camera 260 may be monitored for changes by camerainterface 510. When bus 1020 arrives, camera interface 510 may detectmotion and may, in response to detecting the motion, generate a displayinsert 1030 with a video feed of images captured by top edge camera 260.Additionally, pattern recognition module 520 may analyze the videoimages and may detect a large moving object, which may be identified asa bus based on pattern information stored in pattern DB 525. Thus, userinterface generator 570 may generate a textual and/or audible message,informing the user that a bus is arriving.

While display insert 1030 is shown in FIG. 10 as appearing on top ofapplication icons and/or types of selection objects (e.g., buttons,links, etc.), in other implementations, display insert 1030 may appearbehind the application icons and/or other types of selection objects.For example, display insert 1030 may appear as a background image on thedisplay of mobile communication device 110. In yet otherimplementations, display insert 1030 may be moved by the user by slidingdisplay insert 1030 to a different location in order to accessapplication icons and/or other types of selection objects that arecovered by display insert 1030.

FIG. 11 is a diagram of a second exemplary scenario 1100 according to animplementation described herein. In scenario 1100, a user 1110 issitting in a vehicle and may be looking down at the screen of mobilecommunication device 110. Mobile communication device 110 may detectdashboard camera 1120 based on wireless signals. For example, mobilecommunication device 110 may detect dashboard camera 1120 using aBluetooth discovery process and may establish a Bluetooth connection inresponse. Alternatively, user 1110 may link/couple mobile communicationdevice 110 to dashboard camera 1120. Dashboard camera 1120 may provideimages and/or a video feed to mobile communication device 110 and mobilecommunication device 110 may monitor the detected images and/or videofeed for changes. Mobile communication device 110 may detect motionafter the traffic starts to move and may generate a display insertmessage window 1130 that displays the moving traffic as captured bydashboard camera 1120, along with a message that informs user 1110 thatthe traffic has started to move.

FIG. 12 is a diagram of a third exemplary scenario 1200 according to animplementation described herein. In scenario 1200, a user is waiting tomeet someone for lunch. The user may access a calendar application thatincludes an event associated with a contact in the user's address book.The user may have scheduled a lunch with a contact named “John Smith.”While the user is waiting, the user may receive a reminder 1210 of theupcoming event from the calendar application and may select to bealerted when “John Smith” is arriving.

At a later time, while the user is using and looking down on mobilecommunication device 110, mobile communication device 110 may detectmovement in its environment based on a video feed being from left edgecamera 270. Pattern recognition module 520 may analyze images from thevideo feed and may detect a person in the images. Pattern recognitionmodule 520 may provide the images to environment analysis system 130 andenvironment analysis system 130 may perform facial recognition toidentify the person in the images. Furthermore, environment analysissystem 130 may provide information identifying the person to patternrecognition module 520. Pattern recognition module 520 may determinethat the identified person is John Smith based on information stored ina contacts database on the mobile communication device. In response tothe identification, and based on the alert selected by the user, mobilecommunication device 110 may generate a display insert 1220 that showsimages of the approaching person, along with a message stating that“John Smith is approaching from the left.”

FIG. 13 is a diagram of a third exemplary scenario 1300 according to animplementation described herein. In scenario 1300, a user 1310 may bedriving a vehicle. User 1310 may be sitting in traffic and may be usingmobile communication device 110 while the vehicle is not moving. Mobilecommunication device 110 may detect a vertical orientation and mayselect back camera 250 to monitor the environment within back camerafield of view 1320. Back camera field of view 1320 may include thedashboard of the vehicle.

Pattern recognition module 520 may detect that the user is driving avehicle based on a detected pattern associated with a dashboard. Forexample, pattern recognition module 520 may detect a steering wheelpattern, a speedometer pattern, and/or any other pattern associated witha vehicle dashboard. Once pattern recognition module 520 detects adashboard pattern, pattern recognition module 520 may monitor whetherthe vehicle is moving. For example, pattern recognition module 520 maymonitor the speedometer to determine whether the indicator needle ispointing to a non-zero speed and/or whether a digital speedometerdisplay is displaying a number indicating a non-zero speed. Additionallyor alternatively, pattern recognition module 520 may monitor the speedof the vehicle using another technique, such as by calculating the speedof movement of mobile communication device 110 based on multiple GPScalculations.

In response to determining that the vehicle is moving, a display insertwindow 1330 may be generated to warn the driver that the driver shouldavoid texting and driving. A warning message may discourage the driverfrom using mobile communication device 110 while the vehicle is moving.In some implementations, pattern recognition module 520 may temporarilydisable the display of mobile communication device 110 in response todetecting that the vehicle is moving, in order to prevent the driverfrom using mobile communication device 110 while driving.

In the preceding specification, various preferred embodiments have beendescribed with reference to the accompanying drawings. It will, however,be evident that various modifications and changes may be made thereto,and additional embodiments may be implemented, without departing fromthe broader scope of the invention as set forth in the claims thatfollow. The specification and drawings are accordingly to be regarded inan illustrative rather than restrictive sense.

For example, while series of blocks have been described with respect toFIGS. 8 and 9, the order of the blocks may be modified in otherimplementations. Further, non-dependent blocks may be performed inparallel.

It will be apparent that systems and/or methods, as described above, maybe implemented in many different forms of software, firmware, andhardware in the implementations illustrated in the figures. The actualsoftware code or specialized control hardware used to implement thesesystems and methods is not limiting of the embodiments. Thus, theoperation and behavior of the systems and methods were described withoutreference to the specific software code—it being understood thatsoftware and control hardware can be designed to implement the systemsand methods based on the description herein.

Further, certain portions, described above, may be implemented as acomponent that performs one or more functions. A component, as usedherein, may include hardware, such as a processor, an ASIC, or a FPGA,or a combination of hardware and software (e.g., a processor executingsoftware).

It should be emphasized that the terms “comprises”/“comprising” whenused in this specification are taken to specify the presence of statedfeatures, integers, steps or components but does not preclude thepresence or addition of one or more other features, integers, steps,components or groups thereof.

To the extent the aforementioned embodiments collect, store or employpersonal information provided by individuals, it should be understoodthat such information shall be used in accordance with all applicablelaws concerning protection of personal information. Additionally, thecollection, storage and use of such information may be subject toconsent of the individual to such activity, for example, through wellknown “opt-in” or “opt-out” processes as may be appropriate for thesituation and type of information. Storage and use of personalinformation may be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

No element, act, or instruction used in the present application shouldbe construed as critical or essential to the embodiments unlessexplicitly described as such. Also, as used herein, the article “a” isintended to include one or more items. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise.

What is claimed is:
 1. A method, performed by a handheld portablewireless communication device, the method comprising: obtaining, by thehandheld portable wireless communication device, image data from anenvironment of the handheld portable wireless communication device usingone or more cameras, wherein at least one of the one or more cameras islocated on an edge of the handheld portable wireless communicationdevice, wherein the edge corresponds to a plane perpendicular inrelation to a plane of a display of the handheld portable wirelesscommunication device, wherein the one or more cameras includes multiplecameras, and wherein obtaining the image data from the one or morecameras includes: selecting at least one of the multiple cameras basedon an orientation of the handheld portable wireless communication devicewith respect to a horizontal position; detecting, by the handheldportable wireless communication device, a change in the environment ofthe handheld portable wireless communication device based on theobtained image data; and providing, by the handheld portable wirelesscommunication device, the obtained image data from the one or morecameras in a display insert window on the display of the handheldportable wireless communication device, in response to the detectedchange in the environment.
 2. The method of claim 1, wherein obtainingthe image data from the one or more cameras includes: combining theobtained image data from two or more of the multiple cameras into apanoramic image of the environment.
 3. The method of claim 1, whereinobtaining the image data from the one or more cameras further includes:connecting to a remote camera via a wireless connection.
 4. The methodof claim 1, wherein detecting the change in the environment of thehandheld portable wireless communication device based on the obtainedimage data includes at least one of: detecting motion in the environmentbased on the image data; detecting a change in light intensity in theenvironment based on the image data; or detecting a change in a colorprofile of the environment based on the image data.
 5. The method ofclaim 1, wherein detecting the change in the environment of the handheldportable wireless communication device based on the obtained image dataincludes: zooming in on an area of the environment associated with thedetected change.
 6. The method of claim 1, further comprising:performing pattern recognition on one or more images associated with thedetected change in the environment to identify an object, entity, orevent in the environment; and providing information relating to theidentified object, entity, or event to a user in connection with thedisplay insert window.
 7. The method of claim 6, wherein performingpattern recognition on one or more images associated with the detectedchange in the environment to identify an object, entity, or event in theenvironment includes: extracting one or more features from the one ormore images associated with the detected change in the environment; andidentifying the object, entity, or event in a database stored in thehandheld portable wireless communication device based on the extractedone or more features.
 8. The method of claim 6, wherein performingpattern recognition on the detected change in the environment toidentify an object, entity, or event in the environment includes:providing at least some of the image data to an environment analysissystem; and receiving information relating to the identified object,entity, or event from the environment analysis system.
 9. The method ofclaim 6, wherein performing the pattern recognition on the detectedchange in the environment to identify an object, entity, or event in theenvironment includes: determining that a person is present in theenvironment; performing facial recognition to identify the person;identifying a contact of the user based on the identified person; andproviding an indication to the user that the identified contact is inthe environment, in response to identifying the contact.
 10. The methodof claim 9, further comprising: identifying a calendar event associatedwith the identified contact; and providing an indication of theidentified calendar event to the user, in response to identifying thecalendar event.
 11. The method of claim 9, further comprising: receivinga selection from the user to inform the user if the contact is detectedin the environment.
 12. A handheld portable wireless communicationdevice comprising: one or more cameras located on edges of the handheldportable wireless communication device, wherein the edges correspond toplanes perpendicular in relation to a plane of a display of the handheldportable wireless communication device; and logic configured to: obtainimage data of an environment of the handheld portable wirelesscommunication device from at least one of the one or more cameras,wherein the one or more cameras includes multiple cameras, and wherein,when obtaining the image data from the one or more cameras, the logic isfurther configured to: select at least one of the multiple cameras basedon an orientation of the handheld portable wireless communication devicewith respect to a horizontal position; detect a change in an environmentof the handheld portable wireless communication device based on theobtained image data; and provide the obtained image data from the atleast one of the one or more cameras in a display insert window on thedisplay of the handheld portable wireless communication device, inresponse to the detected change in the environment.
 13. The handheldportable wireless communication device of claim 12, wherein the one ormore cameras located on the edges of the handheld portable wirelesscommunication device include: a first camera located on a left edge ofthe handheld portable wireless communication device; a second cameralocated on a top edge of the handheld portable wireless communicationdevice; and a third camera located on a right edge of the handheldportable wireless communication device.
 14. The handheld portablewireless communication device of claim 12, wherein, when obtaining theimage data from at least one of the one or more cameras, the logic isfurther configured to: connect to a remote camera via a wirelessconnection.
 15. The handheld portable wireless communication device ofclaim 12, wherein, when detecting the change in the environment of thehandheld portable wireless communication device based on the obtainedimage, the logic is further configured to at least one of: detect motionin the environment based on the image data; detect a change in lightintensity in the environment based on the image data; or detect a changein a color profile of the environment based on the image data.
 16. Thehandheld portable wireless communication device of claim 12, wherein thelogic is further configured to: perform pattern recognition on one ormore images associated with the detected change in the environment toidentify an object, entity, or event in the environment; and provideinformation relating to the identified object, entity, or event to auser in connection with the display insert window.
 17. The handheldportable wireless communication device of claim 16, wherein, whenperforming the pattern recognition on the detected change in theenvironment to identify an object, entity, or event in the environment,the logic is further configured to: determine that a person is presentin the environment; perform facial recognition to identify the person;identify a contact of the user based on the identified person; andprovide an indication to the user that the identified contact is in theenvironment, in response to identifying the contact.
 18. One or morenon-transitory computer-readable media, storing instructions executableby one or more processors, the one or more non-transitorycomputer-readable media comprising: one or more instructions to obtainimage data from a camera located on an edge of a handheld portablewireless communication device located in an environment, wherein theedge corresponds to a plane perpendicular in relation to a plane of adisplay of the handheld portable wireless communication device, whereinthe one or more instructions to obtain the image data include: one ormore instructions to select the camera from a plurality of cameras basedon an orientation of the handheld portable wireless communication devicewith respect to a horizontal position; one or more instructions todetect a change in the environment of the handheld portable wirelesscommunication device based on the obtained image data based on at leastone of a detected motion, a detected change in light intensity, or adetected color profile in the environment; one or more instructions toperform pattern recognition on one or more images associated with thedetected change in the environment to identify an object, entity, orevent in the environment; one or more instructions to generateinformation relating to the identified object, entity, or event; and oneor more instructions to provide the obtained image data from the camerain a display insert window on the display of the handheld portablewireless communication device, in response to the detected change in theenvironment, wherein the display insert window includes the generatedinformation relating to the identified object, entity, or event.